Conventionally, there is a technique in which an image processing apparatus captures a marker image, thereby combining a virtual object with a captured image at the position of the marker image in the captured image and displaying the combined image.
The above technique, however, merely displays the virtual object at the position of the marker image in the captured image, and there is room for improvement in the display of an image with enhanced interest.
Therefore, it is an object of an exemplary embodiment to provide an image processing system, an image processing apparatus, and an image processing method that can enhance interest in image processing using a marker image, and a storage medium having stored therein an image processing program that can enhance interest in image processing using a marker image.
To achieve the above object, the exemplary embodiment can employ, for example, the following configurations. It should be noted that it is understood that, to interpret the descriptions of the claims, the scope of the claims should be interpreted only by the descriptions of the claims. If there is a conflict between the descriptions of the claims and the descriptions of the specification, the descriptions of the claims take precedence.
In an exemplary configuration of an image processing system according to an exemplary embodiment, an image processing system includes a first information processing apparatus for causing an image of a predetermined virtual space to be displayed on a first display apparatus, and a second information processing apparatus for causing an image of real space captured by an image capturing apparatus to be displayed on a second display apparatus. The first information processing apparatus includes: one or more processors configured to: acquire a user input; place in the virtual space a marker image that is allowed to cause a predetermined virtual object to appear; in accordance with the user input, change a viewpoint for displaying a part of the virtual space on the first display apparatus; and display on the first display apparatus the virtual space viewed from the viewpoint. The second information processing apparatus includes: one or more processors configured to: acquire a captured image captured by the image capturing apparatus; if the marker image is included in the captured image, combine a virtual object corresponding to the marker image with the captured image at a position corresponding to the marker image in the captured image; and if the virtual object has been combined, display on the second display apparatus the captured image with which the virtual object has been combined.
It is preferable that an image to be displayed on the first display apparatus should be a virtual space image (for example, a panoramic image) having a viewing angle of 180° or more in one of an up-down direction and a left-right direction. More preferably, the image should be a virtual space image having a viewing angle of 360° in one of the up-down direction and the left-right direction. Further, it is preferable that the viewing angle in the other direction is equal to or greater than the viewing angle of the image to be displayed on the first display apparatus. More preferably, the viewing angle in the other direction should be equal to or greater than twice the viewing angle of the image, 120° or more, 150° or more, or 180°. Further, if the virtual space image is a panoramic image, the image may be captured by an image capturing function of the first information processing apparatus, or may be obtained by loading via a predetermined storage medium or a network an image captured by another apparatus having the function of capturing the image.
Based on the above, if having found a marker image in an image of a virtual space displayed on a first display apparatus, a user of a second information processing apparatus captures the virtual space image in which the marker image is displayed, and thereby can cause a virtual object to appear from the marker image in the virtual space image. Meanwhile, a user operating a first information processing apparatus can change a viewpoint for the virtual space image. The user of the first information processing apparatus changes the viewpoint, and thereby can search the virtual space for the marker image. At this time, the user of the first information processing apparatus can change the viewpoint, and therefore can have the experience of feeling as if searching for the marker image while actually walking in the virtual space. This makes it possible to provide unconventional interest to image processing using a marker image.
Further, in accordance with the user input, a position of the viewpoint may be allowed to be changed so that a distance from the viewpoint to the marker image in the virtual space changes.
Based on the above, in accordance with a user input, the distance between the viewpoint and the marker image changes. This makes it possible to provide unconventional interest of a marker image, which requires the operation of bringing the viewpoint close to the marker image to capture the marker image and cause the virtual object to appear from the marker image.
Further, the virtual space may include a plurality of panoramic images associated with spots on a map. In this case, the marker image may be placed at a predetermined position of the at least one panoramic image associated with at least one of the spots. One of the spots on the map may be selected as a current spot based on the user input and the panoramic image associated with the current spot may be selected from the plurality of panoramic images, thereby changing the viewpoint. In the display on the first display apparatus, an image of, in the panoramic image associated with the current spot, a partial area determined in accordance with a user operation may be displayed on the first display apparatus.
Based on the above, the user can change the viewpoint for the virtual space including a plurality of panoramic images associated with spots on a map and search the panoramic images for the marker image. At this time, the user changes the direction of the line of sight, and thereby can look around and move the viewpoint. Thus, the user can have the experience of feeling as if searching for the marker image while actually walking in real space represented by the map. This makes it possible to provide unconventional interest to image processing using a marker image.
Further, in the display on the first display apparatus, if the marker image is placed at another spot that satisfies a predetermined condition with respect to the current spot, an other spot marker image may be displayed together with the panoramic image associated with the current spot on the first display apparatus, the other spot marker image representing the marker image placed at the other spot when viewed from a viewpoint of the panoramic image at the current spot.
Based on the above, if the marker image is placed at another spot that satisfies a predetermined condition with respect to a current spot, an other spot marker image is displayed at the current spot. Thus, even if the marker image is not placed at the current spot, the other spot marker image that satisfies the predetermined condition is displayed, whereby it is possible to notify the user that the marker image is placed at another spot.
Further, the one or more processors of the first information processing apparatus may be further configured to, based on a direction from the current spot to the other spot on the map, determine a placement position of the other spot marker image in the panoramic image associated with the current spot.
Based on the above, it is possible to allow the user to, based on a placement position of the other spot marker image, recognize the direction in which the marker image is placed when viewed from the current spot.
Further, based on a positional relationship on the map between a viewpoint at the other spot and the marker image placed at the other spot, the placement position of the other spot marker image may be determined in the panoramic image associated with the current spot.
Based on the above, it is possible to, based on the placement position of the other spot marker image, represent more accurately the direction in which the marker image is placed when viewed from the current spot.
Further, based on a distance from the current spot to the other spot calculated based on information indicating a position of a spot on the map, the placement position of the other spot marker image may be determined in the panoramic image associated with the current spot.
Based on the above, it is possible to, based on the placement position of the other spot marker image, represent more accurately the distance at which the marker image is placed when viewed from the current spot.
Further, a direction of the other spot marker image when viewed from the viewpoint at the current spot may be determined so as to represent a direction of the marker image placed at the other spot when viewed from the viewpoint at the current spot.
Based on the above, it is possible to cause the direction of the marker image when viewed from the viewpoint at the current spot to coincide with the direction of the other spot marker image. This makes it possible to make the correspondence between the marker image and the other spot marker image more easily understandable.
Further, it may be determined whether or not the marker image is placed at another spot adjacent to the current spot on the map and/or another spot within a predetermined range from the current spot on the map, and if the marker image is placed, the other spot marker image may be displayed together with the panoramic image associated with the current spot on the first display apparatus.
Based on the above, if the marker image is placed at another spot adjacent to the current spot on the map and/or another spot within a predetermined range from the current spot on the map, the other spot marker image is displayed at the current spot. Thus, it is possible to display the other spot marker image at a spot appropriate for the marker image, for example, such that the marker image is placed at another spot adjacent to the current spot.
Further, the panoramic images may be generated by panoramically photographing the plurality of spots in real space. In this case, a virtual image having a predetermined design may be placed on the panoramically photographed panoramic images in a superimposing manner, thereby placing the marker image.
Based on the above, it is possible to save the trouble of generating a panoramic image by placing an object representing the marker image in real space.
Further, the marker image may be placed in the virtual space in accordance with the user input.
Based on the above, it is possible to place the marker image in accordance with a user input. This makes it possible to cause the virtual object to appear in the virtual space at a position desired by the user.
Further, the exemplary embodiment may be carried out in the forms of an image processing apparatus that can be included in the above image processing system, a storage medium having stored therein an image processing program for causing a computer to execute at least some of the above operations, and an image processing method including the above operations.
According to the exemplary embodiment, a user can change a viewpoint for a virtual space image. The user changes the viewpoint, and thereby can search a virtual space for a marker image, which can cause a predetermined virtual object to appear by, if the marker image is included in a captured image captured by an image capturing apparatus, performing image processing on the captured image. At this time, the user can change the viewpoint, and therefore can have the experience of feeling as if searching for the marker image while actually walking in the virtual space. This makes it possible to provide unconventional interest to image processing using a marker image.
These and other objects, features, aspects and advantages of the exemplary embodiments will become more apparent from the following detailed description of the exemplary embodiments when taken in conjunction with the accompanying drawings.